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‘The Legislative Audit Committee 
of the Montana State Legislature: 


This is our information systems audit of the Child Care Under the Big Sky (CCUBS) 
system managed by the Early Childhood and Family Support Division of the 
Department of Public Health and Human Services (DPHHS). 


This report provides the Legislature information about managing return on investment 
and security concerns as CCUBS ages. This report includes recommendations to 
enhance risk identification and remediation procedures to protect CCUBS data. Our 
findings also address the need for DPPHS to develop a modernization strategy that 
includes consistent review of metrics that identify public value. A written response 
from DPPHS is included at the end of the report. 


We wish to express our appreciation to Department of Public Health and Human 
Services personnel for their cooperation and assistance during the audit. 


Respectfully submitted, 
/s/ Angus Maciver 


Angus Maciver 
Legislative Auditor 


Room 160 ¢ State Capitol Building * PO Box 201705 * Helena, MT * 59620-1705 
Phone (406) 444-3122 * FAX (406) 444-9784 * E-Mail lad@mt.gov 
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REPORT SUMMARY 
INFORMATION SYSTEMS AUDIT 19DP-04 DECEMBER 2020 


Mon TANA LecIsLaTive Aupit Division 


Child Care Under the Big Sky (CCUBS) 
System Modernization and Security 
DEPARTMENT OF PUBLIC HEALTH AND HUMAN SERVICES 


The Child Care Under the Big Sky (CCUBS) went live in 
2002. As CCUBS technology ages and becomes obsolete, 
risks to security increase and the system may not be a 
cost-effective solution. By developing a modernization plan 
for CCUBS and continually reviewing it, the agency can 
understand if the value of CCUBS is maintained or if larger 
replacement plans need to be engaged. This also needs 
to coincide with ensuring that the final steps of security 
risk assessment are completed to make sure maintenance 
decisions coordinate with security needs. Improving these 
final steps will ensure high risks within CCUBS are 
addressed in a timely manner. 


KEY FINDINGS: 

The technologies used by CCUBS are becoming obsolete. However 
DPHHS continues to use the system and update it as technologies become 
unsupported. There is no annual review of consistent metrics to identify when 
large-scale replacements are needed because CCUBS is no longer providing an 
acceptable business value. Our analysis shows a recent transition into negative 
return on investment as DPHHS pays for technologies to be updated. 


While risk assessments and security plans are updated annually for 
CCUBS, DPHHS does not have formal procedures in place to ensure 
high risks are eliminated or reduced. We identified a high-risk security 
concern within CCUBS in recurring risk assessments. There were no formal 
remediation plans with timelines and milestones developed when the risk was 
initially identified. Therefore, the risk went unaddressed and was re-identified 
in the next risk assessment. 


RECOMMENDATIONS: 

In this report, we issued the following recommendations: 
To the department: 2 

To the legislature: 0 


(continued on back) 
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For the full report or more 
information, contact the 
Legislative Audit Division. 


leg.mt.gov/lad 


Room 160, State Capitol 
OM sre. A000) 
Helena, Montana 59620 
(406) 444-3122 


The mission of the 
Legislative Audit Division 

is to increase public trust 

in state government by 
reporting timely and accurate 
information about agency 
operations, technology, and 
finances to the Legislature 
and the citizens of Montana. 


To report fraud, waste, or 
Plo thyer 


Online 


www. Montanafraud.gov 


Email 
LADHotline@mt.gov 


rll 

(Cetrayscelo) 

(i010) pe Tomoye 
(solr) 
(406)-444-4446 


Text 
(704) 430-3930 





RECOMMENDATION #1 (page 23): 

Modernization Strategy Guidance 

Develop a modernization strategy that plans around obsolete 
technologies, develops metrics for continual measurement of return 
on investment, and tracks metrics on a yearly basis according to state 


policy. 


Department response: Concur 


RECOMMENDATION #2 (page 30): 

Risk Mitigation Policy 

Develop and implement authorization to operate procedures and 
establish quarterly review of timelines and processes for addressing 
risks. 


Department response: Concur 


Chapter | — Introduction and Background 


Introduction 


The Child Care Under the Big Sky (CCUBS) system went live in 2002. The system 
is maintained by the Early Childhood and Family Support Division (ECFSD) of the 
Department of Public Health and Human Services (DPHHS). Part of the mission 
of ECFSD is to improve the quality, affordability, and accessibility of early care and 
education in Montana. CCUBS supports this mission by facilitating the process that 
provides subsidized childcare to qualified families in Montana and the process for 
licensing childcare facilities. In fiscal year 2019, there were over $25 million in childcare 
scholarships awarded to 5,700 Montana families. The $25 million is a combination 
of the federal childcare development fund, State Special Revenue, and state required 
matching funds. 


CCUBS uses information provided by families to determine subsidy eligibility and 
amount. It also maintains updated information to manage important processes like: 

¢ Paying families childcare grants. 

¢ Childcare provider licensing. 

¢ Family and childcare facility correspondence. 

¢ ~~ Childcare facility compliance and inspections. 

¢ Family and childcare facility reporting. 


¢ Grant awards to childcare facilities. 


Along with storing personal information for each family, CCUBS also stores childcare 
provider information such as training, insurance, number of children in a facility, and 
inspection results and history. CCUBS also interfaces with other public service systems 
to provide and receive information. 


The department has expended approximately $16.5 million since 2011 on a system 
development and maintenance contract for CCUBS with an external vendor. The 
average maintenance cost for the last two years is $1.5 million. Based on concerns 
regarding these costs of maintenance and changing federal requirements and the age 
of CCUBS, the Legislative Audit Committee prioritized an examination of CCUBS. 
This chapter provides background on CCUBS and describes the scope and objectives 


of our audit. 
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CCUBS Main Users 


‘There are two main users of CCUBS: The Early Childhood Services Bureau (ECSB) 
and Child Care Resource and Referral (CCR&R) agencies. Additionally, CCUBS has 


other bureaus and vendors that help manage and support the system. 


ECSB: This bureau falls under ECFSD but specifically oversees childcare licensing 
and inspections and oversees the Best Beginnings Child Care Scholarship Program. 
Staff set up providers in CCUBS and maintain information on each provider such as 
training, insurance, and number of children. They also perform annual inspections of 
childcare facilities. All inspections are tracked and compiled within CCUBS. ECSB 
currently has 34 full-time equivalent staff (FTE). 


CCR&R Agencies: These third-party nonprofit entities work with the department 
through separate contracts. They work on behalf of the department to determine 
eligibility for families seeking childcare subsidies. CCR&R agency staff receive 
information on families from applications and enter the information into CCUBS. 
CCUBS determines the eligibility for each case. The determination includes 
information such as approval status and how much a monthly copay will be. CCR&R 
staff also process subsidy payments by directing state and federal funds to childcare 
providers. The amount paid to childcare providers is based on the number of hours 
of care given and is directly tracked for each child. The map below shows the seven 
different CCR&R regions across Montana. 


Figure 1 
CCR&R Map 
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GRegion 2 DRegion 5 
GRegion 3 Region 6 

















Source: Department of Public Health and Human Services. 





CCUBS Management and Support 

CCUBS is managed by the ECSB and Fiscal and Operations Bureau. They help 
CCR&R agencies with training and system support, and manage the childcare 
subsidies. They are responsible for system administration, such as setting parameters 
and coordinating with the vendor on childcare subsidy application changes. The 
vendor that maintains CCUBS has access to testing systems and provides support 
and help with enhancements to the system. The vendor works with DPHHS'’s Project 
Management Bureau to ensure the enhancements are properly made to the system. 
The State Information Technology Services Division also plays a role in supporting 
CCUBS by providing hosting and infrastructure support. 


Audit Scope and Objectives 


Due to the age of the system, there is an increased risk for security weaknesses, high 
maintenance costs, and a continual need to upgrade the system to meet evolving 
security requirements and federal program changes. Based on these risks, the scope of 
our audit centered around managing obsolescence and security governance. 


Managing Obsolescence: Older systems tend to cost more in maintenance generally 
due to knowledge limitations of aging technologies. Changing federal requirements 
and software upgrades due to the age of the system result in continual enhancements 
as well. The amount of money and effort spent managing these changes in an older 
system increases the potential of producing a negative business value for the agency. 


We reviewed the department's efforts to evaluate the effect of CCUBS’s age on the 
agency. We conducted a return on investment analysis for CCUBS to determine if 
the system still produces a positive value. Using information available, we reviewed 
annual maintenance and upgrade costs from 2011 to present, and determined if the 
system is operating as intended while costs are managed. We reviewed applications 
processed, childcare provider invoicing, and other processes that CCUBS allows 
DPHHS to perform. We also researched other states’ systems like CCUBS to compare 
functionality, costs, and overall structure. 


Security Governance: Security is managed by multiple parties involved in CCUBS. 
Every year the department’s Technology Services Division (TSD), with the support 
of the ECFSD and vendor, assesses the risks to CCUBS. The documentation of 
this assessment is critical to ensure coordination of these parties and remediation or 
acceptance of security risks. DPHHS is a large agency with varying priorities and there 
is high involvement from the vendor for support. Several risks including security role 
designation, risk identification and remediation, and protection of sensitive data are 


increased. 
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Our work focused on reviewing IT roles to determine if they are explicitly identified 
within the risk prioritization process. We reviewed 2018 and 2019 security 
documentation to ensure it is maintained, secured, updated regularly, and contains all 
relevant information. Risk identification and prioritization procedures in these years 
were reviewed to ensure risks are identified and remedied in a timely manner. Current 
user management procedures were reviewed to ensure the agency protects Personal 
Identifiable Information through proper user management procedures. 


From this information we were able to form the following objectives: 


1. Determine if CCUBS is obsolete or still provides public value to the 
Department of Public Health and Human Services. 


2. Determine if CCUBS security governance: 
¢ — Clearly defines responsibility of CCUBS security roles, 


¢ Safeguards personal identifiable information and protected health 
information, and 


¢ Appropriately prioritizes and remedies IT risks in a timely manner 
following IT best practices. 


Audit Methodologies 


To address our objectives, we conducted the following audit work. 


We reviewed best practices for conducting a Return on Investment (ROD): 


¢ ‘The Association for Information Systems published ‘Az Introduction to 
Return on Investment for Information Systems.” This provided information on 
ROI equation and components. ‘The Center for Technology in Government 
at the University of Albany produced the publication Advancing Return on 
Investment Analysis for Government IT; This provided information on public 
value revenue and how to conduct ROIs on systems that do not produce a 
monetary return. 


¢ — State Information Services Division provided documents and guidance that 
would be given to state agencies needing assistance with conducting ROI 
calculations. 


We completed an ROI analysis for CCUBS that required reviewing the following: 
¢ Support, maintenance, and upgrade contract costs. 
¢ Reviewing maintenance and support requests. 


¢ Interviewing users to obtain time estimations on procedures conducted 
within CCUBS. 

¢ — Researched other states’ systems like CCUBS in order to compare system 
functionality and costs. 


We also researched continuous modernization best practices: 


¢ The International Business Machines (IBM) Center for The Business of 
Government provides information on continual modernization in its 2018 
publication A Roadmap for IT “Modernization in Government” and The 
Information Systems Audit and Control Association (ISACA) Journal on 
Continuous Modernization. 


We reviewed state policy and the National Institute of Standards and Technology 
(NIST) industry standards for security governance criteria to determine system security 
requirements and best practices. NIST is a nonregulatory federal government agency 
that develops commonly used security standards and controls for federal agencies. 


We also addressed the controls in place to ensure risk identification and remediation 
occurs for agency systems, user access policies are followed, and security responsibilities 


are defined. To do this, we: 


¢ Reviewed CCUBS security documentation for specific deliverables of the 
risk assessment process and user access management. 


¢ Identified and interviewed personnel responsible for maintaining CCUBS 
security on current risk identification and remediation procedures for 


DPHHS. 


Report Contents 


Our work, findings, and recommendations to the agency are discussed in the following 
chapters: 


¢ — Chapter II describes the ROI we conducted for CCUBS, obsolete technology, 


and the need for a system modernization strategy. 


¢ Chapter HI describes DPHHS annual security reviews, authorization to 
operate, and risk identification and remediation procedures. 
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Chapter Il - CCUBS Requires a 
Comprehensive Modernization Strategy 


Introduction 


As systems age, there can be increases in costs for maintenance and support, technology 
becomes obsolete, and risks arise that can leave systems vulnerable. A technology is 
obsolete when it is no longer efficient, effective, or useful relative to current technology 
that performs a similar function. Examples of this include when part of the system’s 
functionality is no longer supported, stops receiving security updates, better technology 
becomes available to perform similar system functions, or it becomes cumbersome to 


support and upgrade the system. 


As part of our work, we reviewed previous efforts to modernize the Department 
of Public Health and Human Service’s (DPHHS) Child Care Under the Big Sky 
System (CCUBS) and conducted a Return on Investment (ROJ) analysis to get an 
understanding of the current business value of the system. This chapter includes 
information on the ROI analysis we conducted for CCUBS, other state’s structures 
and costs, and continual modernization best practices. 


CCUBS Uses Obsolete Technology 


CCUBS is made of various types of technology that work together to perform necessary 
tasks. Figure 2 shows how these can be broken into three tiers. The client tier refers 
to the front-end technology for the user to interface with, the application tier contains 
the processing logic for data in the system, and the database tier contains the back-end 
technology used to store data. 


Figure 2 
Three-Tier System Architecture 


Client Applications Database 


Pull & 


; Prep Data 
Display 


Data 
Process 
Data 


Source: Legislative Audit Division. 
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CCUBS uses technology that is obsolete and no longer being supported in the 
application tier of the system which is outlined in red in Figure 2 (see page 7). 
While CCUBS is still functioning, it requires a redesign of this tier in order to be 
maintainable, take advantage of new technology, and to continue to integrate with 
DPHHS’s enterprise services. Along with being forced to upgrade CCUBS over the 
years, DPHHS has identified high-level security risks due to a lack of functionality 
within CCUBS. For example, they need to use third-party software to satisfy some 
basic security requirements, such as implementing audit logs. 


In 2014, DPHHS determined that CCUBS was obsolete and requested funding for 
planning activities to replace and redesign the system. DPHHS requested $2 million 
through the (HB) 10 Long-Range Information Technology funding process. The funds 
were requested for performing the planning, request for proposal, feasibility study, and 
business processing analysis and for modeling for the replacement of CCUBS. DPHHS 
submitted the request to the Department of Administration’s State Information and 
Technology Services Division (SITSD) for consideration in the HB 10 funding request 
but it ultimately did not make it into the Governor's budget to be presented to legislators 
during the 2015 Legislative Session. It was unclear why the funding request did not 
make it into the Governor’s budget and there have been no further funding requests 
presented relating to CCUBS. As a result, instead of starting the planning activities for 
replacement, DPHHS has tried to make incremental investments over time to address 


the outdated components of CCUBS. 


Our work focused on analyzing the impact of CCUB’s age and these previous 
maintenance decisions. The following sections discuss state policy for reviewing and 
replacing systems, modernization best practices, and our analysis on what value this 
system is providing the agency in its current state. 


State Policy Requires Annual Evaluation 
of Information Systems 


The SITSD has a policy discussing the process for determining and managing 
obsolescence. The policy states that information technology (IT) systems must be 
reviewed on an annual basis to determine whether they have become obsolete or still 
provide business value. Through this analysis, the agency then must choose how to 
address systems with obsolete technology or functionality that are no longer providing 
business value. State policy does not dictate how an agency should review systems 
or address obsolescence; however, best practices provide guidance for modernizing 
obsolete technology. Modernization focuses on continually upgrading and optimizing 
applications and their underlying infrastructure and services. 


Oncea system is determined to be obsolete, state policy requires a Return on Investment 
(ROD) analysis, or business case be conducted as part of a modernization strategy. A 
business case is a tool used to capture the reasoning for initiating a project or task. 
‘They are needed once the decision to make significant changes to a system is approved. 


An ROI analysis is a financial calculation used to justify the investment in a new 
technology by evaluating current technology and comparing it to the potential 
investment. The state policy regarding these two practices was established in 2017, 
three years after DPHHS’s funding request to plan CCUBS'’s replacement. Therefore, 
there is no documentation available of business need or evaluation of CCUBS from 
2014. Since 2014, DPHHS has maintained CCUBS without reanalyzing obsolescence 
or the public value being provided, as required by state policy. 


ROI Analysis Best Practices Allow Variability 
Depending on the Situation 


Our objective was to conduct an ROI analysis on CCUBS and identify metrics that can 
be used to determine if modernization plans are needed for the system. ROI analysis 
best practices vary widely depending on the situation and there is not a single best 
method to conduct one. Systems that generate revenue use a more direct method of 
determining ROI, while systems that provide a service or value use different methods 
to quantify their value. 


State policy dictates that an ROI analysis be conducted but it does not specify how 
it should be conducted. As a result, we interviewed staff from SITSD in an effort 
to provide insight on how an ROI analysis would be conducted and what resources 
would be made available to state agencies where systems generally provide a public 
value as opposed to generating revenue. SITSD stated that an ROI calculation could 
be done in two different ways after the agency declares the system obsolete: 


¢ — System operation cost vs. public value provided. 


¢ — System replacement cost vs. new public value provided. 


SITSD also provided examples of ROI calculators and multiple sources of information 
and guidance they have for agencies. We used this information along with industry 
best practices to develop our ROI methodology based on system operations costs and 
the current public value provided. 
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Replacing Traditional Revenue Calculation 
With Quantifying the Value of a System 


The equation to calculate ROI is simple. As illustrated by the following graphic, an 
ROL is calculated by identifying the Revenue and Cost of a system: 


ROI = Revenue - Cost of Investment 
Cost of Investment 


However, as it is a system that provides a public service, CCUBS was not intended or 
designed to produce a financial return via revenue generation, so an ROJ analysis must 
rely on other factors that create value from a public perspective. These public values 
are considered the social benefits from the system or other positive benefits of having 
the system in place that do not necessarily relate to generating revenue. The figure 
below shows how public value is generated from information systems processes. These 
processes lead to better agency operations which in turn help citizens and provide 
public value. 


Figure 3 
Public Value Generated 


Information 
Systems 
Processes 


Public Value 
Generated 


Better Agency 


: Direct Citizen Benefit Public-at-Large Benefit 
Operations 


Source: Advancing Return on Investment Analysis for Government IT. 





Consequently, as part of developing an ROI for CCUBS, we had to modify the 
traditional ROI equation to account for a public value factor. The equation below 
incorporates public value rather than revenue. It shows how value and cost can provide 
an idea of the ROI of a system dedicated to providing a public service and it is the 
equation we used for our ROJ analysis of CCUBS. 


ROI= Public Value - Cost of System 
Cost of System 


Adjusting the revenue factor in the equation still provides a valuable metric, especially 
when monitored over time. Through the lifetime of a system, ROI can change, so 
understanding ROI on an annualized basis is helpful for determining current 
investment decisions. ROI also helps decision-makers understand the investment 
overall and if the areas of positive ROI outweigh the areas of negative ROI. 


Applying these changes to an ROI calculation for CCUBS means public value is based 
on its impact on citizen’s ability to receive childcare assistance funding and ensure 
licensed facilities are safe. Our work focused on quantifying this value in a way that 
makes the ROI calculation meaningful to maintenance decisions in CCUBS. 


a 
CONCLUSION 


There are a variety of approaches to determine ROI! for a system. As long 
as evaluations of ROI maintain consistency and include necessary factors 
for either revenues or quantifiable public value, the metric is important in 
modernization strategies. We determined that quantifying public value is the 
best method for determining the revenue of CCUBS. 


Calculating ROI With Public Value 


We were able to obtain information from 2011 to present for public value and costs. 
We used 2011 as a starting point due to not having reliable data from before this point 


in time. 


Due to the variance in how ROI can be calculated, we provided our methodology for 
DPHHS feedback to ensure there was an understanding on both sides of what our 
work would include. DPHHS suggested we capture only processes within the system 
to ensure accurate comparisons. As a result, this is how we structured our ROI analysis 


of CCUBS. 


Based on our research and input from the department, we determined the process for 
conducting ROI would include gathering evidence since 2011 for: 


¢ Costs including support, maintenance, and upgrades. 


¢ — Public value based on procedures within CCUBS only, including the benefits 
generated for processing applications and conducting childcare facility 
inspections. 


Using Procedural Costs to Determine Public Value 


There are multiple ways to determine public value based on the connection between what 
happens in government and the impacts on public stakeholders. The straightforward 
impact of CCUBS includes making specific benefits easier to manage for DPHHS. 
However, the public impact also can include family or community relationships, social 
mobility, and status. There is no data available for these types of impacts at this point, 
but there are resource data to develop a way to understand the value of the system to 
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DPHHS. Resource data, or the personnel costs, are availiable as the money budgeted 
to provide a service to the public. Therefore, they can be seen as the value of the service 


to the public. 


While this approach is conservative in understanding the value of the system and does 
not include the financial benefits obtained by families, it still includes several areas 
of analysis and multiple metrics to develop an understanding of public value. The 
following represent the metrics we reviewed to represent public value for our ROI 
calculation. 


¢ Number of applicants in the system 

¢ — Process application time 

¢ Number of childcare providers 

¢ Time to record childcare provider information within the system 


¢ — System’s ability to handle abrupt change (e.g. COVID-19) 


The following sections discuss our work to identify public values with these metrics, 
what the overall formula represents, and potential solutions gathered from our research 
of other states’ practices. 


Public Value Factors 


Public value factors as defined by best practices fit into four general areas. Table 1 


identifies those areas and how they are applied to CCUBS at a high level. 


Table 1 
ROI Factors Table 





ROI Factors Definition CCUBS Application 


This benefit will allow the company Potential federal fines avoided by 
to avoid a cost completely. using CCUBS. 





Cost Avoidance 





This benefit will reduce (but not CCUBS allowing for quicker 
eliminate) a cost. processes. 


Social benefits of having CCUBS 
Similar to Cost Reduction, but (i.e. more registrations, better 
reducing a capital expense. childcare, data driving policy- 
making decisions). 


Cost Reduction 





Public Framework Revenues 





Maintenance, support, and 


Reoccurring Costs Costs incurred to run system. upgrade contracts. 

















Source: Legislative Audit Division. 





The following sections describes our review of each category and any information we 
were able to use to determine ROI. 


Cost Avoidance 


We wanted to specifically look at the role of CCUBS in maintaining eligibility for 
federal grant funding and what potential penalties CCUBS helps DPHHS avoid. 
DPHHS staff informed us that there are some reports pulled from CCUBS that are 
used in federally required reports. This includes information from CCUBS that is used 
for the Child Care and Development Fund Plan submitted to federal officials. ‘This 
plan describes how federal childcare money is being spent in Montana, how the state 
will maintain a level of effort toward the program, and how any state-matched dollars 
for federal grants will be funded. Maintenance of effort is established by the federal 
program providing the grant and is the financial commitment expected from an 
agency to be eligible for the federal childcare grant. This becomes a type of public value 
because it represents funds further distributed by the state for the benefit of citizens 
and also avoids the potential loss of federal funding. Since the establishment of the 
federal grant, the annual maintenance of effort for Montana has been approximately 
$1.3 million dollars. 


Another cost avoidance factor is the department’s ability to adjust to changing federal 
guidelines. Due to COVID-19, DPHHS was forced to make changes to normal 
processes within CCUBS. Information systems are expected to be agile and respond 
to issues that may arise. Even with the age of CCUBS, DPHHS was able to give 
families and providers a chance to adapt to immediate federal program changes, thus, 
avoiding potential costs for noncompliance and to the public. While there is not a 
value attributed to this, it is important to consider. 


Cost Reduction & Public Value 


The main purposes of CCUBS are to reduce the time and effort of managing federal 
requirements and funds, simplify the process for the public and make assistance easier 
to access. This is where the personnel cost of each transaction becomes the worth of 
the system and is used to quantify its value. By determining the cost per transaction 
or instance, such as entering a paper application into the system, a total value can be 
derived by multiplying by the volume that occurred. For example, if entering in paper 
applications takes 15 minutes, and the employee is paid $13 per hour, the personnel 
cost for each paper application would be $3.25. This value can then be multiplied by 
the total paper applications in a year to determine the annual value of the application 
process. 
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We decided to focus on client intake, eligibility determination, and facility licensing 
activities in order to ensure we covered processes done solely within the system. 
Functionality such as contract management and administrative reviews were not 
included because they do not involve CCUBS functionality, only the data within 
CCUBS. Specific functions we reviewed include: 


¢ Application processing for families seeking subsidized childcare services. 


¢ Authorization of subsidized childcare and recipient case actions and updates 
throughout the year. 


¢ — Processing payments and adjustments to families and providers every month. 


¢ — Establishing a new childcare provider facility in the system. 


To determine average costs for these processes, we gathered time estimations and 
personnel costs from Child Care Resource & Referral (CCR&R) agencie’s staff where 
these processes were taking place. CCR&R agencies are broken into seven different 
regions, so we interviewed department staff from all the regions in our analysis and 
averaged their time estimations. We also used the CCR&R budget information to 
determine the personnel costs for these procedures. We obtained the average hourly 
wage for the system users that manage the processes, namely eligibility specialists from 
CCR&R agencies most recent budget request information from 2016. We averaged 
eligibility specialists’ hourly salaries within each agency, which equated to $13.18. 
Table 2 shows the processes we gathered time information for, and the cost of each 
process based on the hourly salary calculation. 


Table 2 
Calculation for Average Cost Per System Managed Process 





Time Estimations 


by Region (minutes) Average Time | Average Cost 





Processes Per Process Per Process* 


- Paper Application Entry 

















- Online Application Entry 





- Online Invoice | 
_- Adjustments Eas 
































Source: Legislative Audit Division. 


*Based on $13.18 average hourly wage of eligibility specialists across all regions. 





Overall, time estimations from CCR&R agencies were similar but due to different 
amount of staff, number of applications to process, and experience levels at the 
agencies there were some time variances. For example, certain offices had an easier 
time inputting paper subsidy applications. Agencies also noted that time dedicated to 
overpayments or adjustments would vary depending on the severity of the error. 


For our final public value number, we looked at the process of how department staff 
record a new childcare facility, specifically the licensing process within CCUBS. 
Documenting and managing childcare facilities within CCUBS provides value to the 
public by ensuring public safety and helps maintain the quality of licensed facilities. 
We used the same process to determine the average cost of establishing a new childcare 
facility in the system. The average time was estimated at 25 minutes and the average 
salary of staff conducting this is $23.18. Therefore, the average cost per facility is $9.74. 


Overall Public Value Determination for CCUBS 


With individual costs developed for each process, we multiplied those by the 
estimation of occurrences in each year to identify an overall value. Because the number 
of occurrences is not gathered for all processes, we made estimations. The following 
describes how each average cost metric was applied to several occurrences and any 


estimations that were made. 


Online Applications: These applications are tracked within the system, so exact counts 
for online applications per year were used. 


Paper Applications: These application counts are not tracked. The difference between 
the number of online applications and the number of cases that were authorized during 
the year was used to estimate the occurrences. 


Authorization for Subsidy: Each case has to be authorized before receiving a subsidy, 
and time is spent in CCUBS documenting these case events. These are tracked as case 
events within the system. The department provided an estimation for this total for our 
analysis. 


Subsidy Payments: The department produces a management report with various 
family and childcare facility statistics each year. From this report we obtained the 
number of unduplicated families that are enrolled each year since 2016. We used an 
average of this yearly number for the years prior to 2016 when the report did not exist. 
To determine payment occurrences, we multiplied this number by 12 because they 
occur as part of a monthly invoicing process. ‘The invoicing process for these payments 
can happen either online or on paper. We estimated personnel costs for both and used 
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a weighted average of the costs to multiply by the number of occurrences. The weighted 
average was based on 90 percent of invoices being processed online and 10 percent 
on paper. We calculated the weighted average based on 90 percent of invoices being 
processed online (at $0.55 per invoice) and 10 percent on paper (at $1.71 per invoice), 
for a weighted average of $0.61 per invoice processed. 


Overpayments and Adjustments: We estimated a 2 percent error rate in invoicing, so 
2 percent of total invoice estimates were used to determine the number of occurrences. 
This number was used in prior year estimates. 


Establishing New Childcare Facilities: We used the total current facility number that 
the department provided and estimated 10 percent being new each year. This number 


was used in prior year estimates. 


Table 3 shows the yearly calculation of each of these values with the addition of the 
cost avoidance metric for the maintenance of effort related to federal grants. 


Table 3 
Public Value Estimations by Factor and Year 


Public Framework Revenues 








Online 
Applications 


Paper 
Applications 


Authorizations 


Invoice 
Processing 


Adjustments/ 
Overpayments 


Establishing 
New Facility 


State 
Maintenance of 





$ per instance $4.58 $6.77 $6.41 $0.61* $3.30 $9.74 


Effort 





Year 





2011 $0 $20,434 $19,330 $42,554 $4,587 $1,096 


$1,313,990 





2012 $0 $43,253 $40,966 $42,554 $4,587 $1,096 


$1,313,990 





2013 $0 $37,834 $35,860 $42,554 $4,587 $1,096 


$1,313,990 





2014 $82 $36,751 $34,995 $42,554 $4,587 $1,096 


$1,313,990 





2015 $3,808 $36,053 $34,572 $42,554 $4,587 $1,096 


$1,313,990 





2016 $6,906 $36,534 $35,648 $41,681 $4,493 $1,096 


$1,313,990 





2017 $7,963 
2018 $8,526 


$35,342 
$34,746 


$35,283 
$34,578 


$42,862 
$43,610 


$4,620 
$4,701 


$1,096 
$1,096 


$1,313,990 
$1,313,990 





2019 $7,935 $31,474 $31,176 $41,974 $4,524 $1,096 


$1,313,990 





2020 $7,981 $27,688 $26,775 $42,554 $4,587 $1,096 


$1,313,990 























TOTAL $43,201 $340,109 $329,182 $425,451 $45,860 $10,960 


$13,139,900 





Total Public Framework Revenues 


$1,194,763 











TOTAL PUBLIC VALUE 


$14,334,663 





Source: Legislative Audit Division. 


*This is a weighted average of online and paper invoice processing. 


The total of the public value over 10 years areas is almost $15 million, shown in green 
on the table. This represents an estimation of the monetary benefit the public and 
department receive because of CCUB’s processes. 








Identifying Costs Associated With CCUBS 


In addition to assessing the public value associated with the daily administration of 
CCUBS, we also reviewed the following areas to determine CCUBS cost: 


¢ Contract Costs: Support, maintenance, and upgrade costs associated with 


CCUBS. 
¢ System Training: Time spent training users on how to use the system. 


¢ System Support: Focus on support tickets and if waiting on data/system 


fixes affected the productivity of CCUBS. 


While we were able to determine contract costs, system training and support costs 
were not documented in a way that allowed us to determine their costs. We were still 
able to obtain valuable information that can be considered by the agency, which is 
discussed below. 


Contract Costs 


CCUB'’s contract costs were straightforward, in comparison to public value, because 
documenting payments to vendors and determining what funding is necessary year 
to year is a common practice. We decided to use cost information starting in 2011 
due to the availability of information and inflation factors. Total costs for the base 
system and additional features or functions that may not be part of the base but still 
contribute to the business processes managed by CCUBS were included. Since 2011, 
DPHHS spent approximately $16,750,000 for CCUBS maintenance. Additionally, an 
online provider portal was created and cost approximately $410,000. Total contract 
costs totaled $17 million. 


System Training Costs 


Another aspect of cost is system training. This is important because training contributes 
to the efficient use of the system and is essential for the system to be successful. We 
determined that it takes anywhere from two to six months to get specialists fully 
trained to use the system. During this time, eligibility specialist’s work is reviewed 
more thoroughly than the work of fully trained staff. We did not use this number 
in our final calculation because training is dependent on contracted agencies, with 


minimal consistency or formal structure from which to obtain data. 


System Support Costs 


The other aspect of cost of the system we reviewed is support ticket analysis. Our focus 
was support efforts related to the outdated system, which would be manual changes. 
While CCUBS maintenance and support is included in the yearly contract, department 
management need to track this information and quantify how much extra training 
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and support is being spent on these situations. This will also provide information on 


yearly contracting decisions and is an important metric in reviewing the system. 


DPHHS uses a system to track these efforts through support tickets. We were able 
to pull a spreadsheet of support tickets from the last two years of this system. When 
trying to calculate a cost associated with these support 






































tickets and identify trends in extraneous work, we Table 4 

were unable to get specific data for these situations. oo 
Tickets are tracked by when it is created and when it Gost: 
is last updated. A measure of how long the work took ae 
to complete the ticket is not recorded. We found that as 
over the last two years there have been 127 support $2,275 329 
tickets to delete duplicate information such as duplicate $1,597,893 
person or case notes. While a cost value could not be $1,577,893 
established, we were able to identify that these tickets $1,565,290 
took an average of 23 days to be closed. Depending on $1,593,478 
when the error occurred there could be a delay in the $2,315,516 
childcare subsidy process that delays time frames for | [2020 | _$2.275,516_| 











families to receive childcare subsidies. $17,156,750.00 





Source: Legislative Audit 
Division. 


Table 4 shows the yearly breakdown in costs we 
identified. 





Negative Returns Are the Results of Obsolete Technology 


Based on our work described above, we calculated a total cost of $17 million and 
total public value of almost $14 million for CCUBS since 2011. The following figure 
comparea the annual values and costs of CCUBS since 2011. 


Figure 4 


CCUBS Cost and Public Value Comparison by Year 
I 
$1,446,445 I $2,275,329 


$2,315,516 


I 
l 
! $1,577,893 | 
$1,527,333 $1,597,893 2,275,516 
$1,401,990 i $1,565,290 | a 


$1,593,478 
—=Cost 


Value 


$1,214,251 a ata 


$1,214,251 $1,440,347 $1,441,246 


Increase due to additional Increase due to updating obsolete 
functionality technology 


Source: Legislative Audit Division. 


Figure 4 (see page 18) shows that public value has remained relatively the same over 
the years while cost has had two significant increases: one for additional functionality 
where a slight increase in public value also occurred, and another for the transition 
from obsolete technology. ‘The transition from obsolete technology was needed but did 
not result in a change to public value. 


Clearly there is a negative ROI from this time period using this approach. A negative 
ROI in terms of public value means more money is spent on providing a service than the 
value of the service. However, it is not rational for an agency to replace a system based 
just on the ROI provided at one time. As stated in the best practices we researched, 
training, service and support tickets, and other factors need to be considered in this 
decision. Using only ROI can lead to large financial decisions being made with little 
information or reason to support them. ROI over time should be used by DPHHS to 
determine to what level they are willing to let the system go before making significant 


changes to CCUBS. 


Using the ROI number is a way to start referring to IT costs as a factor in the equation 
of how much value the system provides the state. Best practices and state policy indicate 
a continual approach is needed to evaluate whether a system needs to be modernized. 
At some point, the approach of how to conduct an ROI transitions from current 
value and cost to estimated future value and cost. Looking at future costs can help in 
instances where an agency needs to identify best-fit solutions for major replacements. 
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Developing a strategy with either ROI approach leads to more efficient spending and 
an overall better solution for agencies as they encounter aging systems. 


Costs and Structures Vary Across States 


We wanted to find information on other state childcare systems and determine if costs 
were comparable. We reached out to several surrounding states as well other states 
that use different approaches, like cloud services, commercial off-the-shelf systems, 
or systems consolidated with other public services. Our work included interviews 
with staff from Pennsylvania, Utah, Wyoming, and Washington. We talked with 
North Dakota and Colorado officials but were not able to get the same amount of 
information from them from them as from their counterparts in other states. We chose 
these states due to their proximity to Montana and availability. Due to COVID-19, we 
were only able to get limited information from these states, but this information was 
still useful in identifying different options for managing this type of public assistance 
and providing cost and age information of other states systems. 


Figure 5 shows the age and cost information for the systems other states use to manage 


the same processes CCUBS does. 


Figure 5 
Other States 


® 


—_—_= SS Wyoming 

Washington ~ Eligibility System 
re GIDIY oy. 

Eligibility System a Age: 10 years 

Age: Over 25 years ~~ Implementation Cost: Unknown 

Implementation Annual Maint. Cost: $1 Million 

Cost: Unknown 

Annual Maint. Cost: ‘i 

$150,000 © 





Fi 


Licensing System y 


\ 





Age: 3 years ‘Pennsylvania 
Implementation Cost: $450,000 Single Larger System 
\ Annual Maint. Cost: $50,000 Age: 18 Years 
. te basedonlicenses ———_/ | |mplementation Costs: Unknown 





Licensing System y 
Age: 3 years uf 
Implementation /Utah 


Cost: $11.5 million 
Annual Maint. Cost: 
\$1.1 million 





Eligibility System 

Age: 11 Years 

Implementation Cost: Unknown 
Annual Maint. Cost: $805,000 


Licensing System 
Age: 8 years 





Implementation Cost: $300,000 
\Annual Maint. Cost: $160,000 _/ 


Source: Legislative Audit Division. 


(Annual Maint. Cost: $5 Million | 





Due to COVID-19, we were unable to gather enough information for an ROI 
calculation of other states to compare to Montana. However, we were still able to review 
the costs of the various approaches. The following discussion describes the approach 
each state takes: an integrated approach that involves a system managing multiple 
assistance program’s eligibility, benefit or subsidy determinations, and licensing; or 
a multiple-system approach where these functions are separated between individual 
systems. 


Pennsylvania is modernizing a large integrated system that supports an assortment of 
childcare and education services. Pennsylvania is pushing for more commercial off-the- 
shelf systems across the state but currently they are simply updating their integrated 
system. While Pennsylvania is not geographically close, it is a unique example of how 
one state has structured its childcare and licensing system. 


Utah also uses an integrated-system approach for eligibility and subsidy determination, 
but has a separate system for childcare licensing. ‘The eligibility system also includes 
online applications for other public assistance programs like Medicaid and the 
Supplemental Nutrition Assistance Program. 


In contrast to Pennsylvania and Utah, Wyoming uses two systems to manage 
eligibility and licensing. No other assistance programs are managed by the systems. 
The licensing system has web-based functionality and staff can update information in 


real time. 


Washington also uses a separate payment and childcare facility attendance tracking 
systems. The payment system is not dedicated to childcare payments; multiple 
payments are processed for the department. Their licensing system is currently only 
used for childcare, but they are looking to expand to foster care and other areas. 


North Dakota is currently in the process of procuring a childcare licensing data 
system. They are looking at both custom-build and cloud-based solutions with the 
assumption that cloud-based will come in closer to their budget and be their preferred 


route. 


Colorado currently use a commercial off-the-shelf system as their platform for 
Licensing, child care subsidy and attendance, and quality assessment and improvement 
programs. They have contracted with a vendor for the system’s implementation, and 


operation’s and maintenance. 


Our research identified that some states have started upgrades and transitions to new 
systems. However, there are still older systems, like CCUBS, that are being used. 
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Other states have different structures governing their childcare programs, which can 
lead to difficulties in direct comparisons of systems. 


Obsolete Technology and Need for 


Continual Modernization 


Modernization strategies involve a cyclical process of planning, assessment, execution, 
and measurement and tracking. Figure 6 (see page 22) shows the full process for 
determining how to modernize systems. ‘These four steps need to be recurring and based 
on measurements established by the agency. DPHHS has started these processes with 
a yearly IT security review but currently there is a gap in action for a comprehensive 
modernization strategy. The ROI we conducted is an example of a metric used for a 


continual modernization strategy. 





Figure 6 
Modernization Strategy Graphic 
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IBM Center for The Business of Government-A Roadmap for IT Modernization in 
Government. 


Modernization strategies are an ongoing process to allow for continuous improvement 
rather than costlier sporadic “catch ups.” There is a focus on forming a diverse team 
that can identify obsolete technology and how that technology supports mission goals. 
There should be a strong commitment to communication with agency, functional, and 
technical leadership, and key users. Strong communication can help the team develop 
metrics for technology to measure the system’s return on investment or value. For 
example, ROI provides a useful metric to start a dialogue about system obsolescence 
and can provide department staff and legislative policymakers with the information 
they need to consider future large-scale IT investments, such as the potential future 
replacement of CCUBS and other state IT systems. 


Modernization Needs to Be a Proactive Process 
We agree with DPHHS that CCUBS is obsolete, and our ROI analysis shows that 


the return is negative at this point in time. However, we recognize that there are 
difficulties in replacing large information systems from a financial, technical, and 
logistical standpoint. DPHHS has faced these difficulties by trying to update CCUBS 
incrementally over time. DPHHS’s approach has been reactionary in nature. Best 
practice research dictates that modernization must be an on-going process rather than 
a single stand-alone event. 


Continual modernization is required to ensure Montana information systems are 
secure and provide value to Montana citizens. DPHHS needs to review CCUBS on a 
yearly basis to determine what their modernization strategy is. This approach will help 
them determine when it is time to initiate larger projects, such as system replacements. 


a 


RECOMMENDATION #1 





We recommend the Department of Public Health and Human Services 
develop a modernization strategy to address obsolete technologies and 
diminishing return on investment of Child Care Under the Big Sky that 
includes: 


A. Proactive planning to address obsolete technologies, 


B. Develope metrics, like return on investment or scoring, for continual 
measurement, and 


C. Tracking these metrics and reviewing obsolescence on a yearly basis 
according to state policy. 


TO 
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Chapter Ill - CCUBS Security Risk 
Assessment and Management 


Introduction 


Child Care Under the Big Sky (CCUBS) contains Personal Identifiable Information 
(PI) and Protected Health Information (PHI) including demographic and financial 
information for families and providers, as well as disability and immunization 
information on families. Agencies with systems that contain sensitive information, 
such as CCUBS, need to ensure they meet security requirements to protect that 
information. These security controls are outlined within state policy. 


We found the Department of Public Health and Human Services (DPHHS) has 
established a security program that they use to conduct an annual risk assessment 
within CCUBS. However, improvements can be made to ensure plans are documented 
to remedy or reduce high risks in a timely manner. This chapter contains information 
related to annual security reviews, authorization to operate, and risk identification and 
remediation procedures. 


Security Documentation Notes Key System Risk Decisions 


One key control in managing security is the risk assessment process and development 
of high-level security documentation. High-level security documentation helps keep 
security responsibilities and processes organized for IT systems. The following figure 
shows the main purpose of the documents and when they are needed during the risk 
assessment process. 


Figure 7 
Risk Assessment Process 


Risk Assessment Process Key Documentation 


Risk Identification System Security Plan (SSP) 


Review of entire technology environment to identify Requires periodic review and modification based on 
potential risks evolving risks and changing controls 


A 4 vy 





Risk Remediation imatclamelmAreit(o)alr-lare mV Il(=¥<)elal=\¥l (aa OND) 


Identification of current or needed controls to reduce Outlines which risks need to be addressed first and 
risks should contain plans for remediation 


Risk Acceptance PNUiiate)ar4-1i(e)am cen ©) e\-1¢-1(-m (AKO) 


Official sign off on the use of the system and to 


Management's approval of risk levels accept any identified risks and remediation plans 





Source: Legislative Audit Division. 
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These documents also help manage security at a large agency where multiple systems 
have risks that need to be prioritized. There are also multiple divisions and vendors that 
need to coordinate, and in CCUBS’s case, the vendor helps manage a large portion of 
the development and maintenance processes. Due to these factors, there need to be 
detailed procedures outlined in department policy to ensure updates are appropriately 
conducted and risks are addressed and remediated in a timely manner. 


We reviewed these procedures relative to CCUBS and identified that the security 
program is mostly complete with standard procedures and clear roles and responsibilities. 
However, improvements in addressing identified risks are needed. The following 
section describes DPHHS’s security program and areas of needed improvement. 


DPHHS Has Established Security 
Roles and Responsibilities 


Defined security roles for an application, especially with substantial vendor involvement, 
help ensure security responsibilities are clear. It is important to have established security 
roles to ensure contractor accountability. State policy guides agencies on IT security 
management and specific security roles that need to be identified. State policy states 
that agencies must have a system security plan (SSP) in place that provides an overview 
of the security requirements for the system, describes security controls in place, and 
documents security categorization of the system. 


DPHHS has worked to establish their security program over the last few years. We 
reviewed the most recent SSP from 2018 to understand the progress made and identify 
key personnel for interviews, and we identified two levels of security roles pertaining 
to CCUBS, including those responsible for high-level security administration and 
decisions, and those responsible for day-to-day operational security of CCUBS. High 
level security administration includes the chief information officer and the security 
compliance officer. These roles engage in the creation and approval of the SSP and Plan 
of Action and Milestone (POAM). Responsibilities at the operational level include 


system testing, maintenance, and managing user access. 


Through multiple interviews with staff and review of documentation, we identified 
that DPHHS employees know and understand their security roles. These roles align 
with state policy and help keep CCUBS secure. 


CCUBS’s Security Program Controls User 
Access but Lacks Monitoring Activity 


We focused the security program review on user management and event logging due to 
the nature of information within CCUBS. User access management is the procedures 
that prevent unauthorized access and event logging is the procedures and controls in 
place to identify unauthorized activity. 


User Access Management: We reviewed user access management procedures, 
including how users are created, maintained, and reviewed. Overall, DPHHS policy 
and procedures for user access maintenance and review follow National Institute of 
Standards and Technology (NIST) and state policy. These key controls ensure that 
user accounts meet the following requirements: 


¢ — Specific request for access forms are filled out and approved by the Technology 
Services Division security team. 


¢ New users are reviewed and approved by system administrators and 
supervisors. 


¢ — Roles assigned align with individuals job functions. 
¢ 6-month reviews are done of all users. 


¢ Specific request-for-termination forms are filled out when an individual 
leaves the agency or no longer needs access. 


User Activity Monitoring: While the procedures to ensure users are given accurate 
access and to maintain appropriate access exist, procedures need to be in place to 
monitor activity of users, especially those with elevated access. For example, vendor 
access often needs further scrutiny and increases the need for additional controls. The 
CCUBS vendor has elevated access to the system, which introduces additional risk. To 
limit vendor-related risk, systematic audit logs are required by state policy to document 
activity and to be monitored by the system owner. These logs provide a baseline of 
system activity and can help detect when abnormal events occur. Audit logs provide 
information on: 
¢ Users 
Actions are automatically recorded and tied to them 
Confirm users are properly using the system 
¢ Reconstruction of Events 
| When, how, who, and what happened during the incident 


¢ Intrusion Detection 


Help identify suspicious behavior 
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According to the 2018 and 2019 POAM document developed internally by DPHHS, 
such audit logs are not in place for CCUBS. 


DPHHS Has Taken Steps to Address Security 
Risks at the Organizational Level 
DPHHS has recognized the lack of implementing audit logs for CCUBS as a high 


risk. Department personnel indicated they have been working on implementing state 
security policy over the last five years in order to increase the security around their 
applications and data. They stated higher level security priorities have been completed, 
including overall individual application security assessments, implemented multi- 
factor authentication, implemented a vulnerability scanning program, and building 
the ability to conduct risk assessments annually. 


DPHHS also indicated that increasing security resources and staff have been a slow 
process over the last four years. This has made conducting annual risk assessments on 
all systems annually a challenge. Another challenge in addressing audit logs within 
CCUBS is the solution for implementing them. While this is identified as a high risk, 
we are able to discuss it in this report because of compensating controls already in 
place at the agency. CCUBS was developed almost 20 years ago, and due to its age 
DPHHS needs to use additional software to integrate with CCUBS and provide the 


necessary controls. 


However, it is important to address this situation in CCUBS due to the risk of not 
being able to identify suspicious or unauthorized activity. Without proper monitoring, 
it makes it difficult for security staff to monitor and identify suspicious behavior within 
the system. Information like multiple unsuccessful log-in attempts, data changes, or 
parameter changes are key metrics of such activity. These are common indicators of 
unauthorized activity that puts the payment and personal information of Montana 
citizens at risk. 


Final Risk Assessment Procedures Need 
Action Plans and Timelines 


While there are challenges in removing high-level risks within CCUBS that must be 
balanced with high-level risks found in other applications, application-specific risks 
that are high need to be removed as soon as possible. As these risks go unaddressed, 
the system and its data are more vulnerable to threats. Identifying and remediating 
high risks found in CCUBS ensures that it is secure and plays an important role in 
authorizing its use. 


According to state policy, the department CIO must review high-level security 
documents, noted in Figure 7 (see page 25), and issue an Authorization to Operate (ATO) 
every two years. An ATO is the official management decision to authorize operation 
of an information system and to accept risks found from a review process. The POAM 
document that DPHHS creates contains risks categorized as high, medium, and low. 
High-level risks need to have a remedy plan in place to either lower its risk or remove it 
completely. State policy dictates that these action plans be reviewed quarterly. 


Currently, DPHHS has an ATO policy. This policy identifies individuals responsible 
for the ATO process, but it lacks timelines and action plans to remedy high risks 
documented in the POAM. Each risk identified in the action plan would go through a 
different change process, depending on severity. However, staff indicated that they do 
not currently have written procedure in place for the POAM risk action plans. 


When reviewing the ATO process we identified that various procedures to ensure 
accountability, like timelines, reviews, and follow-ups are not defined. DPHHS does 
not have procedures in place to ensure high risks are eliminated or reduced or a way to 
monitor the progress of risk elimination plans to ensure authority to operate timelines 
are met. They are currently developing the final steps of the process and plan to have 
it completed in time for next year’s assessment cycle. During fieldwork, DPHHS also 
updated the current POAM to include steps and dates to ensure that high risks are 
remedied within a 60-day time frame. 


Authorization to Operate and Risk Mitigation Plan 
Procedures Need Accountability and Follow-Through 


While high-level procedures for risk identification are important to establish, individual 
high risks to systems are also important to manage in a timely manner. An example of 
high risk that has been identified and not addressed is the audit logs within CCUBS. 
Until CCUBS has audit logs in place, unauthorized activities occurring within the 
system cannot be identified. Users with elevated access rights can potentially change or 


misuse system information. 


While DPHHS has recently formalized the prioritization of risks to the department 
and individual systems identified in security assessments, creating formal procedures to 
ensure that ATO timelines and requirements are met will keep the system functioning 
safely and minimize risks, like those posed by not having audit logs. 
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RECOMMENDATION #2 





We recommend the Department of Health and Human Services improve risk 
mitigation policy by: 


A. Developing and implementing Authorization to Operate procedures that 
include documented risk acceptance or procedures and timelines for 
remedying or reducing high risks, and 


B. Establish a quarterly review of timelines and processes for addressing 
risks to ensure actions are completed. 
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Angus Maciver 

Legislative Auditor 

Office of the Legislative Auditor 
State Capitol, Room 160 
Helena, Montana 59620-1705 


Re: Child Care Under the Big Sky LAD Audit Recommendations 
Dear Mr. Maciver: 


The Department of Public Health and Human Services has reviewed the Performance Audit 
of Child Care Under the Big Sky completed by the Legislative Audit Division. Our 
responses and corrective action plans for each recommendation are provided below. 


Recommendation #1; 
We recommend the Department of Public Health and Human Services develop a 
modernization strategy to address obsolete technologies and diminishing return on 
investment of CCUBS that includes: 

a. Proactive planning to address obsolete technologies, 

b. Develop metrics, like return on investment, or scoring, for continual measurement of 

system return on investment or business value, and 
c. Track these metrics and review obsolescence on a yearly basis according to state 


policy. 


Response: Concur 


Corrective Action: 

The department will develop a modernization strategy for CCUBS that will evaluate the 
technology being used against the business value being gained and cost. The strategy 
will include establishing an annual review of obsolescence and metrics to ensure the 
application is still providing value. 


Planned Completion Date: 07/31/2021 


Recommendation #2: 
We recommend the Department of Public Health and Human Services improve risk 
mitigation policy by: 


a. 


Developing and implementing Authorization to Operate procedures that include 
documented risk acceptance or procedures and timelines for remedying or 
reducing high risks, and 

Establish a quarterly review of timelines and processes for addressing risks to 
ensure actions are completed. 


Response: Concur 


Corrective Action: 

The department will update the risk mitigation policies and procedures to include 
documented risk acceptance or timelines for remedying and/or reducing high risks. The 
department will review the status on a quarterly basis to ensure the actions are completed. 


Planned Completion Date: 03/31/2021 


Sincerely, 


cc: 





Laura Smith, Deputy Director/Economic Securities Branch Manager 

David Crowson, Chief Information Officer 

Jamie Palagi, Early Childhood and Family Services Division Administrator 
Chad Hultin, Audit Liaison 


